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ABSTRACT 


Traditionally, data mining algorithms and machine learning 
algorithms are engineered to approach the problems in isolation. 
These algorithms are employed to train the model in separation on a 
specific feature space and same distribution. Depending on the 
business case, a model is trained by applying a machine learning 
algorithm for a specific task. A widespread assumption in the field of 
machine learning is that training data and test data must have 
identical feature spaces with the underlying distribution. On the 
contrary, in real world this assumption may not hold and thus models 
need to be rebuilt from the scratch if features and distribution 
changes. It is an arduous process to collect related training data and 
rebuild the models. In such cases, Transferring of Knowledge or 
transfer learning from disparate domains would be desirable. Transfer 
learning is a method of reusing a pre-trained model knowledge for 
another task. Transfer learning can be used for classification, 
regression and clustering problems. This paper uses one of the pre- 
trained models — VGGFace with Deep Convolutional Neural 
Network to classify images. 
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1. INTRODUCTION 

A Facial Recognition System is a technology that 
can capture a human face anywhere in an image or a 
video and also can find out its identity. A Face 
Recognition system has proven to be very beneficial 
in case of user identity verification in the recent past 
replacing the age-old authentication mechanisms like 
password protection or the One Time Passwords. In 
the last decade or so, we have seen a huge growth in 
the smart mobile industry for using face verification, 
and also numerous apps like Snapchat or Instagram 
which can put interesting filters on face. The ongoing 
research in this field has come up with a lot of scope 
in a wide range of applications, such as surveillance 
systems or law enforcement. 


This paper is about exploring a popular algorithm 
running behind a Face Recognition system. 
Specifically, we are going to discuss the Siamese 
Networks. The Paper assumes that the reader has a 
basic understanding of Machine Learning and Deep 


(http://creativecommons.org/licenses/by/4.0) 


Learning technologies and has some idea in building 
Neural Networks with Tensor Flow. 


2. BACKGROUND 

The term originally comes from the conjoined twin 
brothers Chang and Eng Bunker (May 11, 1811 — 
January 17, 1874), who were the first pair to be 
known internationally. The term is used for those 
twins who are physically connected to each other at 
the chest, or at the abdomen or the pelvis. The two 
individuals were originally from Thailand, formerly 
known as Siam, hence the name. The Neural Network 
we are going to see in this article also consists of a 
pair of Networks which are actually the same, hence 
the name derives from the Siamese Twins. 


3. PROPOSED METHODOLOGY 

As we saw above, the Siamese twins are connected 
physically, the Siamese network also consists of a 
pair of Neural Networks which are identical to each 
other, also known as Sister Networks Unlike a 
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conventional CNN, the Siamese Network does not — weights and the biases in such a way that it should 
classify the images into certain categories or labels, | produce a smaller distance between the two images, 
rather it only finds out the distance between any and if they belong to different labels, then the 
two given images. If the images have the same label, distance should be larger. 

then the network should learn the parameters, i.e. the 


Network 1 


nbedding Laye 128 


Network 2 


Fig 1 Architecture of a Siamese Network 
As it shows in the diagram, the pair of the networks are the same. The Siamese Network works as follows. 


To train a Siamese Network, a pair of images are picked from the dataset, each one processed by one of the 
networks above. (In next few sections, we will see how to generate pairs of images from the dataset.) 


The networks have the same structure, hence the same operations will be performed on the respective images. 


The Neural Networks at the end have Fully Connected Layers, with the last one consisting of 128 nodes. This 
layer is the final feature that gets produced when the network is applied on the image. It’s called the Embedding 
Layer Representation. So the two images in the pair processed by the Siamese Network produce two different 
Embedding Layer Representations. 


The Network then finds the Euclidean distance between both the embedding layers. If the images are of the 
same person, then it is expected that the embeddings will be very similar, hence distance should be smaller. 
However, if the images are of different people, then the distance is expected to be a higher value. A Sigmoid 
Function is applied on the distance value to bring it to 0-1 range. 


A loss function is put on the sigmoid result, to penalize the network to update its weights and the biases. We are 
using Binary Cross Entropy in this paper for the loss function. Updation of the weights and the biases done on 
both the networks are exactly the same. 


This process repeats for all the image pairs generated from the dataset. The same approach described above is 
put into code below. We have used Tensor Flow APIs to build the network architecture. 


4. RESEARCH METHODS 

importnumpyasnp 

importmatplotlib.pyplotasplt 

importcv2ascv 

importos 

importsys 
fromtensorflow.keras.layersimportInput,Dense, Activation 
fromtensorflow.keras.layersimportFlatten 
fromtensorflow.keras.modelsimportModel 
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importtensorflowastf 
fromtensorflow.keras.lossesimportBinaryCrossentropy 
fromtensorflow.kerasimportbackendask 
fromtensorflow.keras.optimizersimportAdam 
fromtensorflow.keras.layersimportActivation 
fromdeepface.basemodelsimport VGGFace 
fromdeepface.basemodelsimportFacenet, VGGFace 
vgg=VGGFace.baseModel() 
vgeg.load_weights(’../input/weightsss/vgg_face_weights.h5') 
https://www.kaggle.com/raafaq/weightsss/download 
size=224 
shape=(size,size,3) 
cnt=0 
forlyerinvgg.layers: 
ifent>34: 
lyer.trainable=True 
lyer.activation=tf.keras.layers.Activation(‘tanh’) 
else: 
lyer.trainable=False 
cnt=cnt+1 
“By applying tanh activation function from layer 34, we can later use multiple images of single person for 
generating features and these features will be different . So it will be really helpful in face verification.” 
vgeg_face=Model(inputs=vgg.layers[0].input,outputs=vgg.layers[-2].output) 
defcreate_model(): 
inputs=Input(shape) 
#inputs = BatchNormalization(axis= -1)(inputs) 
outputs=vgg_face(inputs) 
x=Flatten()(outputs) 


model=Model(inputs,x) 
print(vgg_face.summary()) 
returnmodel 

extractor=create_model() 
imgA=Input(shape=shape) 
imgB=Input(shape=shape) 
featA=extractor(imgA) 
featB=extractor(imgB) 
defeuclidean_distance(vectors): 
(featA,featB)=vectors 
sum_squared=k.sum(k.square(featA-featB),axis=1,keepdims=True) 
returnk.sqrt(k.maximum(sum_squared,k.epsilon())) 
distance=Lambda(euclidean_distance)([featA,featB ]) 
print(distance) 
outputs=Dense(1,activation="sigmoid" ,name='ddd')(distance) 
model=Model(inputs=[imgA,imgB],outputs=outputs) 
model.summary() 
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on generating pairs of images of same and different persons labeling same face image as | and different face as 
0. We can train the euclidean layer to maximize the distance when the pair of faces is of same person otherwise 
minimize the distance. 


5. RESULTS 

The evaluation metric used in this experiment is accuracy. While training, we can see that oou model performed 
very accurate. By changing activation funtion and making the layers trainable our VGGFace transfer learning 
model achieved 99.42% accuracy on validation set. The results are given below: 


Q_ Search 4 
Face recognition using VGGFace transfer learning i 


Notebook Data Logs Comments(0) Settings qq: 


-} 
@ ; def preprocess_pairs(images, labels, classes): < 
targets = [j for j in range{len{classes})] 
Y label_indices = {} 
for label in targets: 
ini label_indices.setdefault{label, [index for index, curr_label in enumerate{labels) if label == curr_label]) 


compreher 


= pamages = [] 
plabels = [] 
a for i, image in enumerate{images): 
pos_indices = label_indices.get{labels[i]} 
Vv pos_image = images[np.random.choice{pos_indices) ] 
pimages.append{{image, pos_image)) 
plabels.append{1) 


neg_indices = np.where{labels != labels[i]} 
neg_image = images[np.random.choice{neg_indices[@] }] 
pimages.append{{image, neg_image}) 
plabels.append{a) 
rm} return np.array{pimages), np.array{plabels) ¥ 


Figure 2 Generating image pairs 


After no. of epochs the loss, accuracyis given as... 


Q 4 
Face recognition using VGGFace transfer learning 


an qa & 


Figure 3 Model fit function and few recorded epochs 
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Face recognition using VGGFace transfer learning 


Notebook Data Logs Comments (0} Settings @ i GY Gale 
@ < 
g 8839 - accuracy: 995 val_loss: @.0735 - val_accuracy: 6.9942 
El .@837 - accuracy: 6.996: val_loss: @.072@ - val_accuracy: @.9942 
<> : : 6.8888 - accuracy: 8.998 val_loss: @.071@ - val_accuracy: @.9942 
& 3/ : 6.87 accuracy: 6.999% val_loss: @.0698 - val_accuracy: 8.9942 
B Sf mere accuracy: @.997 val_loss: @.0698 val_accuracy: @.9942 
wv 3/ 4 (a). accuracy: @. val_loss: @.0676 - val_accuracy: @.9942 
accuracy: @.9985 - val_loss: @.4666 - val_accuracy: @.9942 
86/88 

43/43 - 11s : 8.8727 accuracy: @.998 val_loss: @.0656 - val_accuracy: 98.9942 

<tensorflow.python.keras.callbacks.History at @x7fea@8Gad35a> 
i . 


Figure 4loss, accuracy after 80 epochs 


The corresponding curves of losses and accuracies were plotted: 


Q Search & 


Face recognition using VGGFace transfer learning 


A Notebook Data Logs Comments(0) Settings 1 
® Bit. plot (np.arange{nn) , keyss[ ‘loss'],color='green', label='loss' ) < & 
plt.legend{) 
bY plt.plot(np.arange({nn},keyss[‘val_loss'],color='black', label = '‘val_loss' )} 
plt.legend{) 
ini} plt.plot(np.arange({nn),keyss['accuracy'],color='red',label = 'accurcy') 
plt.legend{ ) 
2 plt.plot(np.arange{nn),keyss['val_accuracy'],color='purple',label = 'val_accu') 
plt.legend{) 
ei plt.xlim(@, 88) 
Q plt.ylim(9,1) 
plt.show() 
Vv 
rn} + 


Figure 5 loss, accuracy vs epochs plot 


The related notebook can be found here Click here. 
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